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I INTRODUCTION (U) 

(U) Since 1973, when the investigations of the human information-accessing capability 
called remote viewing (RV) first began at SRI International, 1 * evaluating the quality of the 
information obtained has been a continuing challenge. In order to develop valid evaluation 
procedures, two basic questions must be addressed: 


(1) What constitutes the target? 

(2) What constitutes the response? 

1 If the RV task is research-oriented, the targets are known, and therefore can be 

precisely defined. In -oriented tasks, however, the targets are generally unknown 

and their descriptions are problematical. In both task domains, RV responses tend to consist of 
sketches and written phrases. A method to encode unambiguously this type of “natural 

language " is one of the unsolved problems in computer science, and there has been little progress 

~ 

to date. Thus, a complete definition of an RV response is also problematical. 


\ An on-oriented RV task poses further problems. High-quality RV does 

not always provide useful For example, the RV may provide additional support for 

information that has been verified from other sources, but provide no new information. In some 
cases, however, an overall low-quality RV may provide key elements that positively influence an 
analyst’s interpretation. 

Another characteristic of current laboratory analysis techniques is that they do 
not provide an a priori assessment of the RV quality. While this is not a problem in the 
laboratory, ; _ , applications require such evaluation. An RV analyst cannot provide 

ratings from the RV alone; rather, the analyst must provide a priori 
probabilities that individual RV-response elements (or concepts) are present at the target site. It 
remains the responsibility of an analyst to determine whether such data are ultimately 

useful. 


* Analysis of laboratory RV has been a major part of the ongoing Cognitive 

Sciences Program. 2 ' 7 For FY 1989, we focused on the development of a prototype analysis 
system that would provide the needed a priori assessments for tasking^" ^ 



U) References are at the end of this report. 
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II METHOD OF APPROACH (U) 

4 The analysis of remote viewing (RV) data in an environment differs 

considerably from laboratory analysis. Most often, analysts have incomplete or no information 
about the target site and are required to provide a priori assessments of data gathered from RV 
sessions. In this section we outline a prototype analysis system for > RV that uses 

concepts from fuzzy set theory, historical archival data, and “templates” of typical; 
targets. In addition, we apply this prototype system to an existing target pool as an illustration of 
the power of the technique. 

A. (U) Fuzzy Set Formalism 

A A more com P let e description of the full fuzzy set formalism can be found in our 
literature. 4 ' 7 For the purpose of this report, we have summarized that formalism in general terms 
that are not specific to either laboratory experiments or tasking. 

1. (U) Construction of Target and Response Fuzzy Sets 

(U) A formal definition of a target and its associated RV response (i.e., the data 
obtained from an RV session) is necessary to any analysis system. To use the fuzzy set method, a 
universal set of elements is constructed on which target and response descriptions are based. 
These elements should contain descriptive aspects of the target material and incorporate items 
that typify responses from the intended viewers. This universal set should also be extendible 
(i.e., allow for additional items that may arise in the responses). - 

(U) In general, the task of an RV analyst is to assign a membership value (*t) 
between 0 and 1 to each element in the universal set. The numerical value for each element in a 
response is assigned by the degree to which the analyst is convinced that the given element is 
present in that response. Membership values for target elements are assigned on the basis of the 
degree to which the elements contribute to the target description. 

| | In the laboratory, the targets are known, so that defining a universal set of 

elements is comparatively straightforward. 4 ' 7 In <tfflic<chW| tasks, however, defining a single 
universal set of elements that is appropriate for all operations is difficult. Because the usual 
'task is so highly mission-dependent, defining a single universal set of elements that is 
customized to that mission becomes easier. 


r~ 
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The OppU'&fiari analyst, as opposed to an RV analyst, should construct such 
a list for each mission. While there may be considerable similarities between element lists for 
different missions, undoubtedly the lists will require specialization. In Section II-C below, we 
show the construction of one element list and how it can be applied to a set of 65 | _ 

[ targets. 

2. (U) Analysis of Complete Responses 

| Once an appropriate universal set of elements has been created, and fuzzy 

sets that define the target and the response have been specified, the comparison between them is 
straightforward. We have defined accuracy as the percent of the target material that is described 
correctly by a response. Likewise, we have defined reliability (of the viewer) as the percent of 
the response that is correct. 4 Although in the laboratory it is required to provide a posterior 
probability estimates of the target-response match, in an setting, this may be less 

important. All that is usually necessary is to describe the accuracy and reliability for complete 
responses, and for individual target elements of interest. These quantities for the y'th sessions are 


2 w ‘< r j n tju 

k a 1 

n 

2 WkR j .* 

*= 1 

n 

X w k (Rj n Tj) k 

k = l 

n 

£ « , « 7 m 

k = l 


a) 


( 2 ) 


where the sum over k is called the sigma count in fuzzy set terminology, and is defined as the sum 
of the membership values (ji) for the elements of the response, the target, or their intersection, 
and n is the number of possible elements as defined by the element list. A fuzzy intersection is 
defined as the minimum of the intersecting fuzzy set membership values. In this version of the 
definitions, we have allowed for the possibility of weighting the membership values, Wk, to 
provide mission-defined relevances. 

(U) For the above calculation to be meaningful, the membership values for the targets 
must be similar in kind to those for the responses. For most mission-dependent specifications, 
this is generally not the case. The target membership values represent the degree to which a 
particular element is characteristic of the target, and the response membership values represent 
the degree to which the analyst is convinced that the given element is represented in the 
response. 
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(U) Until RV abilities can encompass the recognition of elements as well as their degree of 
target characterization, we are required to modify the target fuzzy set. An analyst must decide 
upon a threshold above which an element is considered to be completely characteristic of the 
target site. In fuzzy set theory, this is called an a-cut: a technique to apply a threshold to the p 
values such that if the original value exceeds it, reassign the value to 1, otherwise set it to 0. In 
this way, the analyst’s subjectivity can be encoded in the response fuzzy set, and Equations 1 and 
2 remain valid. 


3. (U) Analysis of an Individual Element 


(U) Equations 1 and 2 can be simplified to provide an accuracy and reliability on an 
individual element basis instead for a complete response. For example, let N be the number of 
sessions against different targets that exist in a current archive for a specified viewer. Let e be an 
element in question (e.g., airport). Then the empirical probability that element e is in the target, 
given that the viewer said it was, is given by 



( 3 ) 


where Nc is the number of times that the individual was correct, and Nr is the number of times 
that element € was mentioned in the response. R(e) is also the reliability of the viewer, for that 
specified element. 

(U) To compute what chance guessing would be, we must know the occurrence rate 
of element e in the N sessions. Let N 0 be the actual number of times element e was contained in 
the N targets. Then the chance-guessing empirical probability is given by 

i 


Ro( e ) can also be considered as the guessing reliability (i.e., the reliability that would 
be observed if the viewer guessed e during every session). The more R(e) > R 0 (e), the more 
reliable the individual is for the specified element. 

* (U) The empirical probability that th^e viewer said element e, given that it was in the 
target, is given by 

Nc_ 

N o' 



A(e) is also the accuracy of the viewer for that specified element. 

(U) As a numerical example, suppose a single viewer participated in N = 25 sessions. 
Let e = “airport.” Further suppose that N 0 = 5 of the targets actually contained an airport. 
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(U) 

Then, R 0 (airport) = 0.20 is the chance probability (i.e., guessing airport during every session 
would only by 20 percent reliable). Assume that the! viewer mentioned airport Ni = 6 times and 
was correct Nc - 4 times. Then this viewer’s reliability for airports is computed as R(airport ) = 

0.67 > Ro(airport) = 0.20. The viewer’s accuracy fo? airports is computed as A(airport) = Nc/N 0 

! 

= 0.80. Thus in this example, we can conclude th?t this viewer is reasonably accomplished at 
remote viewing an airport. 

1 

j 

2 

B. (U) Prototype Analysis System 

\ ! We assume that an > ___ • analyst has constructed a mission-dependent 

universal set of elements. We further assume that there are a number of competing 
interpretations of the target site in question. j 

i 

1. (U) Target Templates 

The first step in our prototype ’analysis system is to define templates (i.e., 
general descriptions of classes of target types) of all competing target interpretations from the 

universal set of elements. I ' ~* 

__ > 

I > Exactly what the templates should represent is entirely dependent upon 

what kind of information is sought. Both the underlying universal set of elements and the 
templates must be constructed to be rich enough to allow for the encoding of all the information 
_ That is, if neither the set of elements nor the templates can meaningfully 
represent information abouti ~t\\& __j < then it will be unreasonable to consider 

asking, qucs.-lib'iS a.bt>vA the site. Furthermore, a certain 

amount of atomization is necessary because such division into small units provides the potential 
for interactions within the universal set of elements. If the profile of a l ^facility consists of a 
single element, the template would be useless unless the response directly stated that particular 
element; rather, the profile should be constructed from groups of elemental features 

_ P 

\ There are two different ways to generate target templates. The most 

straightforward technique is also likely to be the most unreliable, because it relies on the analyst’s 
judgment of a single target type. With this method, the analyst, who is familiar with the 
intelligence problem at hand, simply generates membership values for elements from the 
universal set of elements based upon his or her general knowledge. Given the time and 
resources, the best way to generate template membership values is to encode known targets that 
are closely related Each template ji is the average value 

across targets, and thus is more reliable. If it is known that some targets are more 
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i 

"characteristic" of the target type than others, then a weighted average should be computed. In 
symbols, 



k * l 

V * f 

hi 


( 4 ) 


where the sums are over the available targets that constitute the template, uik are the target 
weights, and the pj.k are the assigned membership values for target k. 


2. (U) Archival Database 

' : A critical feature of an analysis system fo^ t RV data is that along 

with the current RV data to be evaluated, the individual viewer's past performance on an 
element-by-element basis must also be included. For example, if a viewer has been relatively 
unsuccessful at recognizing i facilities, then a| , reference in the current data should not 
contribute much in the overall analysis. 

I As ground truth becomes available for each session, a performance database 

should be updated for each viewer to reflect the new information This database should be a 
fuzzy set whose membership values for each element are the reliabilities computed from 
^Equation 3. 

3, (U) Optimized Probability List 

r \ The 8 oal of any [ >RV analysis system is to provide an a priori 

prioritized and weighted list of target possibilities that results from a single remote viewing that is 
sensitive to the performance history of the viewer. Assuming that a template exists for each of 
the possible! .interpretations, an analyst should adhere to the following protocol: 

(1) Analyze the RV data by assigning a membership value (p) for each element iifthe 
universal set of elements. Each p represents the degree to which the analyst is 
con vinced that the particular element is included in the response. ~ . 


(2) Construct a crisp set, Rc, as an at — cut of the original response set. By adopting a 
threshold of 0.5, for example, then the resulting crisp set contains only those 
elements that the analyst deems most likely as being present in the response. 

(3) Construct an effective response set, Re, as Re = Rc u Re, where Re is the reliability set 

drawn from the archival database.! \ 
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(4) Using this effective response set, compute an accuracy and reliability in accordance 
with Equations 1 and 2. Then compute a figure-of-merit, Mj, for the jth competing 
interpretations as 

Mj — cij x r j « 

Of course, the accuracy and reliability use the effective response set from step 3 
above. 

(5) Order the Ms from largest to smallest value. Since the figures— of— merit range in value 
from 0 to 1, they can be interpreted as relative probability values for each of the 
alternative target possibilities. 

H'- 

By following such a protocol, an analyst can produce a list of target alternatives that is sensitive to 
the current remote viewing yet takes into consideration to the individual viewer's archival record. 

, y 

C. (U) Partial Application of Analysis System to Existing Target Pool 

(U) We have used an existing target pool (developed under a separate program) as a test 
bed for the analysis system described above. 

1. (U) Criteria for Inclusion in the Target Pool 

1 ~ 

Targets in this pool have the following characteristics: 

• Each target is within an hour and a half automobile drive of SRI International. 

• Each target simulates df" — Kite t ,>> interest. 

• Each target fits generally within one of five functional categories: Production, 
Recreation, Scientific, Storage, and Transportation. 

Each target meets a consensus agreement of experienced RV monitors and 
analysts about inclusion in the pool. 

(U) The pool consists of 65 targets. Initially, they were divided into 13 groups of five 
targets each, where each group contained one target from each of five functional categories. By 
carefully organizing the targets in this way, the maximum possible functional difference of the 
targets within each group was ensured. Table 1 shows a numerical listing of these targets. 
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Table 1 


. “znsformer Station 
1 Icloark 
1 _mi'lite Dish 
- "->apons Storage 
5 e-al Fleet 
t 1-vel Quarry 
_-mming Pool 
£. Z-ervatory 
c man 

1C Imping and Receiving 

11 L-=snhouse 

12 -mic Area 

II Imilite Dishes 

1 * mrt Warehouse 

15 cai Air Station 

16 Ipr Refinery 

1" -". mound 

1 S -mrium 

19 l~_n Yard 

20 -mm ft 

21 2=wae Treatment Plam 

22. hm-er Tower 

UNCL.-IUTTED 


(U) Numerical Listing of Targets 


23. Space Capsule 

24. Coastal Battery 

25. Bay Area Rapid Transit 

26. Salt Refinery 

27. Candlestick Park 

28. Solar Observatory 

29. Food Terminal 

30. Pedestrian Overpass 

31. Electrical Plant 

32. White Plaza 

33. Space Shuttle 

34. Coastal Battery 

35. Train Terminal 

36. Sawmill 

37. Pond 

38. Wind Tunnel 

39. Grain Terminal 

40. Submarine 

41. Cogeneration Plant 

42. Park 

43. Linear Accelerator 

44. Dump 


45. Pump Station 

46. Ice Plant 

47. Caves/Cliffs 

48. Bevatron 

49. Barn 

50. Golden Gate Bridge 

51. Modern Windmills 

52. Bay lands Nature Preserve 

53. Gas Plant 

54. Auto Wreckers 

55. Fishing Fleet 

56. Radio Towers 

57. Vineyard 

58. Pharmaceutical Laboratory 

59. Toxic Waste Storage 

60. Airport 

61. Car Wash 

62. Old Windmill 

63. Nuclear Accelerator 

64. Reservoir 

65. Train Station 


2 (U) Fuzzy Set l.lcment List 

| In FY W89, we developed a prototype analysis system for analyzing targets 
anc -menses in^ ~"*"|i;emote viewings. A list of elements, based on target function (i.e., 

the zmion specification), is arranged in levels from relatively abstract (information poor) to the 
rela; = ■ complex (information rich). Having levels of elements is advantageous in that each can 
be w=_med separately in the analysis. 

(U) This universal set of elements (included as Appendix A) represents primary 
elemsm in the existing target pool of 65 targets. The set was derived exclusively from this 
know; mget pool. In an actual RV session, however, a viewer does not have access to the 
elemm -it, and thus is not constrained to respond within its confines. An accurate RV analysis 
mus: -cmde any additional data that may be provided in the response; therefore, additional 
spac; izs been provided on the analysis sheets (see Appendix A) to include elements that are 
pan :: response but not initially included as pan of the universal set. 
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(U) We used the technology cluster (i.e., number 4 in Figure 1) to apply Equation 4 
to construct a technology target template. Table 2 shows the targets in this cluster, where the 
horizontal lines indicate the subclustering within the technology group shown in Figure 1. 
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Table 2 

(U) Technology Cluster 


Target 

Name 

56. 

Radio Towers 

1 . 

Transformer Station 

51. 

Modern Windmills 

31. 

Electrical Plant 

41. 

Cogeneration Plant 

3. 

Satellite Dish 

13. 

Satellite Dishes 

8. 

Observatory 

28. 

Solar Observatory 

58. 

Pharmaceutical Laboratory 

63. 

Nuclear Accelerator 

43. 

Linear Accelerator 

48. 

Bevatron 


UNCLASSIFIED * 

(U) Table 3 shows those elements that met or exceeded average membership values 
of 0.4 using Equation 4. 


Table 3 

(U) Principal Elements Contained in the Technology Template 


Levels 

Number 

Name 

Affiliation 

i 

Commercial/Private 

Function 

14 

Research/Experimentation 

Attribute 

24 

Energy 

Modifier 

47 

Electricity/Radio 

Objects 

88 

High Technology Electronics 


99 

Restricted Access 


120 

W^ires/Cables 

Abstract 

mm 

Activity— Passive 


wEm 

Ambiance— Indoor 


131 

Ambiance— Manmade 


137 

Ambiance— Outdoor 


149 

Size— Medium 
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(U) As a self-consistency check, we included the technology template in the total 
target pool and recalculated the clusters. As expected, the technology template was included 
within the subgroup of targets 3 and 13, and well within the technology cluster as a whole. 

D. (U) General Conclusions 

. The goal of this effort was to develop an analysis system that would prove 
effective in providing a priori assessments oft ^remote viewing tasks. If the proper 

mission-dependent universal set of elements can be identified, then, using a viewer-dependent 
reliability archive, data from a single remote viewing can be used to prioritize a set of alternative 
target templates so as to chose the most likely one for the mission. 
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UNIVERSAL SET OF ELEMENTS FOR ANALYSIS OF FUNCTION (U) 
(This Appendix is completely UNCLASSIFIED) 
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AN ANALYST’S GUIDE TO THE UNIVERSAL SET OF ELEMENTS (U) 

A. (U) Introduction 

(U) This appendix is intended to assist an analyst in using the universal set of elements 
shown in Appendix A. We developed six levels of elements ranging from relatively abstract 
(information poor) to the relatively complex (information rich). 

B. (U) Element Levels and Their Use 

(U) The task of the analyst is to assign a membership value between 0 and 1 to each 
individual element. For targets, a numerical value will be assigned on the basis of the presence 
or absence of each element in terms of functional importance. For responses, the numerical 
value will be assigned on the basis of the degree to which the analyst is convincedfthat the 
element is contained in the response. 

(U) All subsequent commentary is referenced by the element numbers in Appendix A. 
Although each level may contain a number of elements, only those individual elements that may 
need explanation are listed below. 

!• (U) Element Level— Affiliation 

(U) Affiliation represents an advanced level of remote viewing functioning. 
Although we infrequently observe this advanced functioning, the data are valuable, and, 
therefore, are included. Elements in this level can be assigned membership values by asking the 
question, “Who owns the target?” There are only three “affiliation” elements: 

(1) Commercial/Private. 

(2) Government: Federal, state, or local governmental ownership (e.g., municipal 
utilities), but excluding military. 

(3) Military, military ownership as separate from the above governmental ownership 
(e.g., a Navy submarine). 
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2 . (U) Element Level— Function 

(U) “Function* also represents an advanced level of remote viewing functioning, and 
it may represent the most important information with regard to overall function. Elements are 
assigned membership values by asking the question, “What is (are) the primary function (s) of the 
target?” There are 14 “function” elements, and a few require further explanation: 

(6) Distribution: the primary function is to receive and to transmit something (e.g., an 
electrical transformer station). 

(8) Extraction: as in the extraction of minerals from the ground. 

(11) Reception: the primary function is only to receive (e.g., a satellite tracking station). 

(13) Refining: the primary function is to refine a raw material into an intermediate or 
finished product (e.g., a saw mill). 

(16) Transmission: the primary function is only to transmit (e.g., a radio tower). 

3. (U) Element Level-Attributes 

(U) “Attributes” can be thought of as clarification for the “function” level. 
Elements are assigned membership values by asking a question similar to, “If the function of the 
target is production, then what is being produced?” There are 20 “attribute” elements, and the 
following require further explanation: 

(18) Animals: animals only . 

(20) Biology: the study of living things in general. 

(21) Chemistry: also includes chemicals. 

(23) Ecology: symbiotic systems in nature, as in ecological zones (e.g., the Bay Lands 
Nature Preserve). 

(24) Energy: energy in a broad sense that also includes radio waves. 

(29) Nature /Natural: general natural objects (e.g., plants and animals). 

(32) Plants: plants only. 

(33) Space exploration: general, includes all experimentation done in space. 

Elements 18 and 32 are given a membership value if the target/response is specifically oriented to 
one item. Otherwise element 29 should be assigned a value. 

t 

4. (U) Element Level— Modifiers * 

(U) “Modifiers” can be thought of as a clarification of the “attributes” level. 
Elements are assigned membership values by asking a question similar to, “If the function of the 
target is production, and vehicles are being produced, then what kind of vehicles are they?” 
There are 36 "modifiers” elements, and only element 66 requires further explanation: 
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(66) Symbiotic: symbiotic relationships not subsumed under natural or ecology (e.g., a 
cogeneration plant). 

5. (U) Element Level— Objects 

(U) “Objects” contains specific elements not necessarily related to function. 
Elements are assigned membership values on the basis of the presence or absence of each object 
in terms of functional importance. There are 47 “objects” elements, and the following require 
further explanation: 

(77) Catwalk: elevated walkway. 

(79) Coastline: used only as coastline of an ocean. 

(88) High-Technology Electronics: silicon-based technology. 

(95) Port/Harbor: port should be marked as in port of departure (e.g., airport, train 
station, seaport). 

(116) Water-Bounded: only completely bounded bodies of water (e.g., pool or pond). 

(117) Water-Canal: manmade. 

(118) Water-Large Expanse: the San Francisco Bay should be marked as a large 
expanse. 

(119) Water-River: also includes stream. 

* 

6. (U) Element Level— General/Abstract Items 

(U) This level contains the most abstract elements. There are 31 elements, and the 
following require further explanation: 

(121) Activity- Active: predominant visually active (e.g., an accelerator is very active 
electromagnetically, but would be considered passive, because there is little visual 
activity); potential activity is considered as passive. 

(122) Activity-Passive: predominant visually passive (e.g., a ballpark is passive most of 
the time) . 4 

(123) Activity-Flowing (Water, Air, etc.): can be natural (e.g. creek) or manmade. 
(128) Ambience-Dangerous: perceived and/or physically dangerous. 

(140) Colorful: to be used only if especially characteristic. 

(141) Modern: to be used only if especially characteristic. 

(142) Odd/Surprising: to be used only if especially characteristic. 

(143) ^)ld: to be used only if especially characteristic. 

(144) Personnel-Few: 1 to 10 employees mostly full-time. 

(145) Personnel-Many: 10 to 1000 employees mostly full-time. 

(146) Personnel-None: no full-time employees, but occasional human attention is 
allowed. 

(148) Size-Large (University Campus): represents a “campus” size area. 

(149) Size-Medium (Building): size of typical single buildings. 

(150) Size-Small (Human): typically, the size of a human (i.e., 6 feet) 

(151) Dull: to be used only if especially characteristic of the color. 
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